Moving picture decoding method and moving picture encoding method

ABSTRACT

High-quality video is provided using a small amount of coded bits. The moving picture decoding method performs inter-frame prediction processing. With the aforementioned inter-frame prediction processing, blocks with similar motion vectors from among the motion vectors in multiple blocks that have already been decoded are combined and a combined area is computed. A predicted vector for a target block to be decoded is computed using the motion vector of the aforementioned combined area, and a motion vector for the aforementioned target block is computed based on the aforementioned predicted vector and a difference vector which is included in a coded stream that is input. A predicted image is generated using the aforementioned motion vector, and a difference image which is included in the aforementioned coded stream and the aforementioned predicted image are added to generate a decoded image.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/874,955, filed Jan. 19, 2018, which is a continuation of U.S.application Ser. No. 15/383,421, filed on Dec. 19, 2016, which is acontinuation of U.S. application Ser. No. 13/132,426, filed Jun. 23,2011, which is a National Phase of International Application No.PCT/JP2009/006438, filed Nov. 27, 2009, which claims priority fromJapanese Patent Application No. 2008-308113, filed Dec. 3, 2008, thedisclosures of which are expressly incorporated by reference herein.

TECHNICAL FIELD

The present invention relates to a moving picture decoding technique fordecoding a moving picture, and a moving picture encoding technique forencoding a moving picture.

BACKGROUND ART

There has been disclosed a technique for predicting images to be encodedin block units using image information in which encoding processing hasbeen completed and encoding a predicted difference between each of theimages and an original image, thereby reducing the amount of coded bitsby removing redundancy of a moving picture. However, the result of ablock search is required to be encoded as a motion vector in addition tothe predicted difference, and overhead of the amount of coded bitsoccurs.

H. 264/AVC (Non Patent Literature 1) has disclosed a predictiontechnique for each motion vector to reduce the amount of coded bits forthe motion vector. That is, when a motion vector is encoded, a motionvector of a target block is predicted using each of encoded blockslocated around the target block, and a difference (difference vector)between the predicted vector and the motion vector is variable-lengthencoded. The accuracy of prediction of each motion vector is however notsufficient. There is a problem that a large amount of coded bits isstill necessary for motion vectors as to images complicated in motionparticularly as in the case where plural moving objects exist.

CITATION LIST Non Patent Literature

Non-Patent Literature 1: ITU-T Recommendation H.264/AVC

SUMMARY OF INVENTION Technical Problem

In the above technique, a large amount of coded bits is still requiredfor each motion vector because the accuracy of prediction of the motionvector is not sufficiently high.

The present invention has been made in view of the above problems. Anobject of the present invention is to improve a method for calculating aprediction vector to thereby reduce the amount of coded bits for eachmotion vector and improve compression efficiency.

Solution to Problem

In order to address the above problems, one embodiment of the presentinvention may be configured as described in Claims, for example.

Advantageous Effects of Invention

High-quality video can be provided with a small amount of coded bits.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram of an image encoding device according to thepresent invention.

FIG. 2 is a block diagram of the image encoding device according to thepresent invention.

FIG. 3 is a block diagram of an image decoding device according to thepresent invention.

FIG. 4 is a block diagram of the image decoding device according to thepresent invention.

FIG. 5 is a conceptual explanatory diagram of inter prediction atH.264/AVC.

FIG. 6 is a conceptual explanatory diagram related to a predictiontechnique for motion vectors at H.264/AVC.

FIG. 7 is a conceptual explanatory diagram related to a predictiontechnique for motion vectors in embodiments 1, 2 and 3.

FIG. 8 is a conceptual explanatory diagram related to a predictiontechnique for motion vectors in the embodiments 1, 2 and 3.

FIG. 9 is a conceptual explanatory diagram related to a predictiontechnique of motion vectors in the embodiment 2.

FIG. 10 is a conceptual explanatory diagram related to the predictiontechnique for motion vectors in the embodiment 3.

FIG. 11 is a conceptual explanatory diagram related to a predictiontechnique for motion vectors in an embodiment 4.

FIG. 12 is a flowchart of an image encoding method according to thepresent invention.

FIG. 13 is a flowchart of an image decoding method according to thepresent invention.

FIG. 14 is a flowchart of the image encoding method according to thepresent invention.

FIG. 15 is a flowchart of the image decoding method according to thepresent invention.

FIG. 16 is a flowchart of the image encoding method according to thepresent invention.

FIG. 17 is a flowchart of the image decoding method according to thepresent invention.

FIG. 18 is a flowchart of the image encoding method according to thepresent invention.

FIG. 19 is a flowchart of the image decoding method according to thepresent invention

DESCRIPTION OF EMBODIMENTS

Embodiments of the present invention will hereinafter be explained withreference to the accompanying drawings.

FIG. 5 conceptually shows the operation of an inter prediction processbased on H.264/AVC.

In H.264/AVC, encoding based on block units is performed to a targetframe for encoding in accordance with raster scan order. Upon executionof the inter prediction, a decoded image of an encoded image included inthe same video (501) as the target frame (503) is assumed to be areference frame (502). A block (predicted or prediction image) (505)high in correlation with a target block (504) in the target frame issearched from within the reference frame. At this time, a differencebetween coordinate values of both blocks is encoded as a motion vector(506) in addition to a prediction difference calculated as thedifference between both blocks. On the other hand, a procedure oppositeto the above may be performed upon decoding. The decoded predictiondifference is added to the block (prediction image) (505) in thereference frame to thereby enable the acquisition of a decoded image.

In H.264/AVC, a prediction technique for each motion vector has beenintroduced to reduce overhead of the amount of coded bits due to themotion vector described above. That is, when a motion vector is encoded,a motion vector of a target block is predicted using an encoded blocklocated around the target block, and a difference (difference vector)between the predicted vector and the motion vector is encoded. Since themagnitude of the difference vector concentrates on approximately 0 atthis time, it is subjected to variable length coding, so that the amountof coded bits can be reduced.

FIG. 6 conceptually shows a method for calculating a prediction vector.Encoded blocks adjacent to the left, upper and upper right sides of atarget block (601) are assumed to be a block A (602), a block B (603)and a block C (604) respectively. Motion vectors MV in the respectiveblocks are assumed to be MVA, MVB and MVC respectively. At this time, aprediction vector is represented as a median among MVA, MVB and MVC.That is, the prediction vector PMV is computed like (605) using afunction Median in which a median value is returned to respectivecomponents of a vector designated as an argument. Further, a differencevector DMV is calculated as a difference (606) between the motion vectorMV of the target block and the prediction vector PMV. Subsequently, thedifference vector DMV is variable-length encoded. Upon decoding, aprocedure opposite to the above may be performed. The decoded differencevector DMV is added to a prediction vector PMV calculated by a proceduresimilar to the above to thereby decode a motion vector MV.

In H.264/AV as described above, the introduction of the predictiontechnique into the motion vectors makes it possible to greatly reducethe amount of coded bits necessary for each motion vector. In the caseof H.264/AVC, however, only adjacent blocks are taken into considerationwhen the prediction vector is computed. It cannot be always said thatthe motion of an object has been reflected. Therefore, particularly whenplural moving objects exist in an adjacent area, the prediction accuracyof motion vectors was not enough and the motion vectors still needed tohave a large amount of coded bits. In an aspect of the presentinvention, motion vectors close in value to each other, of motionvectors in an encoded area are combined to estimate the existence of amotion area and its range. A prediction vector is calculated based on amotion vector of the motion area that exists in the vicinity of a targetarea, thereby making it possible to improve the prediction accuracy forthe motion vector.

A method for calculating a prediction vector PMV, according to thepresent invention will be explained below.

A procedure for calculating the prediction vector PMV is the same as onthe encoding and decoding sides. On the encoding side, a process forcomputing a difference vector DMV between a motion vector MV and aprediction vector PMV and encoding it is carried out. On the decodingside, a process for adding the prediction vector PMV to the decodeddifference DMV and thereby decoding the motion vector MV is carried out.

Embodiment 1

The embodiment 1 uses information about a combined area when aprediction vector of a target block is computed. That is, when thecombined area exists around the target block, it is determined that thetarget block also includes a part of the same object as that included inthe combined area. The prediction vector is calculated based on themotion vector of each block included in the combined area.

Thus, since the prediction vector can be calculated based on a motionvector of a motion area that exists in the vicinity of a target area, itis possible to improve prediction accuracy for the motion vector.Further, the method of calculating the prediction vector is improved toreduce the amount of coded bits for the motion vector and enable animprovement in compression efficiency.

FIG. 7 is a diagram conceptually showing one example of a predictionvector PMV computing method according to the present embodiment.

Consider, for example, where a block (703) including a part of an object(702) is inter-encoded within a frame (701) in which the object (702) isreflected. In this case, motion vectors MV close to each other of motionvectors MV (704) of encoded areas which are located on the left of thetarget block (703) and thereabove, are first combined. When, forexample, a combining condition |V1-V2|<Threshold is established, twomotion vectors V1 and V2 are combined and included in the same area.

Now, |A| is assumed to be the magnitude of a vector A, and Threshold isassumed to be a constant. The constant Threshold may be set to a uniquevalue in advance. Alternatively, values may be freely set on theencoding side and included in a coded or video stream. Further, theconstant Threshold may be determined dynamically based on codinginformation about dispersed values of motion vectors MV, an averagevalue thereof and the like, the magnitude of a combined area, etc. Theconstant Threshold may not comply with this equation in particular.Further, information other than the motion vectors MV, such as colorinformation may be taken into consideration upon combining.

Generally, since motion vectors MV of an area including a part of thesame object have the property of indicating values close to each other,the presence of the object (702) and its range can be specified byacquisition of a combined area (705).

In an aspect of the present invention, the information about thecombined area (705) is utilized when a prediction vector PMV of thetarget block (703) is computed. That is, when the combined area (705)exists around the target block (703), the target block (703) is alsodetermined to include a part of the same object as the object (702)included in the combined area (705). The prediction vector PMV iscalculated based on motion vectors MV of blocks included in the combinedarea (705). Although irrespective of the computing method of theprediction vector PMV in particular, the prediction vector PMV may becalculated as an average value (AVE (C)) of the motion vectors MV in thecombined area as in the case of, for example, an equation (706) of FIG.7. Alternatively, the prediction vector PMV may be computed by selectinga typical motion vector MV in the combined area, such as an intermediatevalue of the motion vectors MV in the combined area.

Here, C is assumed to be a set of the motion vectors MV included in thecombined area, and |C| is assumed to be the number thereof.

A description will next be made of a case where no combined area (705)exists around the target block (703). When the combined area (705) doesnot exist around the target block (703), a prediction vector PMV iscomputed in accordance with a procedure shown in FIG. 8, for example.Now where in a frame (801) in which an object (802) similar to that ofFIG. 7 is reflected, a block (803) away from the object (802) isinter-encoded. In this case, motion vectors MV (804) of encoded areasare combined by a procedure similar to FIG. 7 to obtain a combined area(805). The prediction vector PMV is computed by a conventionalcalculation formula (807) that uses motion vectors MV of blocks aroundthe target block (803), such as H. 264 or the like (806). This ishowever not limited to H. 264 as the conventional method.

The configuration and operation of a moving picture encoding deviceaccording to the present embodiment will next be explained.

FIG. 1 shows one example of the moving picture encoding device accordingto the present embodiment.

The moving picture encoding device according to the present embodimentincludes an input image memory (102) which holds an input original image(101), a block divide unit (103) which divides image data inputted fromthe input image memory (102) into plural areas, an intra prediction unit(105) which performs intra prediction in block units, a motion vectorcalculation unit (104) which performs a motion vector calculation toimage data inputted from the block divide unit (103), an interprediction unit (106) which performs inter prediction in block unitsusing motion vectors MV detected by the motion vector calculation unit(104), a mode selection unit (107) which determines a predictionencoding means (prediction method and block size), a subtraction unit(108) which generates residual data, a transform unit (109) whichperforms transformation to the residual data, a quantization unit (110)which quantizes the residual data inputted from the transform unit(109), a variable length coding unit (111) which performs variablelength coding to the data quantized by the quantization unit (110), aninverse quantization unit (112) which inversely quantizes the residualdata inputted from the quantization unit (110), an inverse transformunit (113) which inversely transforms the data inversely quantized bythe inverse quantization unit (112), an addition unit (114) whichgenerates a decoded image using the inversely transformed data, and areference frame memory (115) which holds the decoded image therein.

The operations of the respective component parts of the moving pictureencoding device according to the present embodiment will be explained.

The input image memory (102) retains a single image of the originalimage (101) as a target frame for encoding and outputs it to the blockdivide unit (103). The block divide unit (103) divides the originalimage (101) into plural blocks and outputs the same to the motion vectorcalculation unit (104), the intra prediction unit (105) and the interprediction unit (106). The motion vector calculation unit (104) computeseach motion vector MV of a target block using a decoded image stored inthe reference frame memory (115) and outputs the same to the interprediction unit (106). The intra prediction unit (105) and the interprediction unit (106) perform an intra prediction process and an interprediction process in block units. The mode selection unit (107) refersto the results of the intra prediction process and the inter predictionprocess and thereby selects a prediction encoding means optical toeither thereof. Subsequently, the subtraction unit (108) generatesresidual data by the optimal prediction encoding means and outputs thesame to the transform unit (109). The transform unit (109) and thequantization unit (110) perform transformation such as DCT (DiscreteCosine Transformation) and quantization processing to the input residualdata respectively in block units and output the so-processed data to thevariable length coding unit (111) and the inverse quantization unit(112). Here, they may perform processing using DST (Discrete SineTransformation), WT (Wavelet Transformation), DFT (Discrete FourierTransformation), KLT (Karhunen-Loeve Transformation), etc., in additionto the DCT. The variable length coding unit (111) performs variablelength coding to residual information expressed by transformcoefficients, and information necessary for decoding such as predictiondirection data used when intra prediction is performed, moving vectorsMV used when inter prediction is performed, etc., based on theprobability of occurrence of codes like the assignment of a short codelength to a code high in the frequency of its occurrence and the like tothereby generate a coded or video stream. The inverse quantization unit(112) and the inverse transform unit (113) perform inverse quantizationand inverse transformation such as IDCT (Inverse DCT) to thepost-quantization transform coefficients to obtain residual data andoutput the same to the addition unit (114). The addition unit (114)generates a decoded image and outputs the same to the reference framememory (115). The reference frame memory (115) stores the generateddecoded image therein.

The inter prediction unit (106) will be explained using FIG. 2.

The inter prediction unit (106) includes a motion vector memory (201)for storing a motion vector MV of each encoded area, a prediction vectorcalculation unit (202) which calculates a prediction vector PMV usingthe motion vector MV of each encoded area, a subtraction unit (203)which calculates a difference between the motion vector MV and theprediction vector PMV to calculate a difference vector DMV, and aprediction image creation unit (204) which generates a prediction image.

The operations of the respective component parts of the inter predictionunit (106) will be explained.

The prediction vector calculation unit (202) calculates a predictionvector PMV of a target block based on each of the motion vectors MV ofthe encoded areas, stored in the motion vector memory (201). Thesubtraction unit (203) calculates a difference between the motion vectorMV calculated by the motion vector calculation unit (104) and theprediction vector PMV to calculate a difference vector DMV (205). Theprediction image creation unit (204) generates a prediction image (206)from the motion vector MV and its corresponding reference frame. Themotion vector memory (201) stores the motion vectors MV therein.

FIG. 3 shows one example of a moving picture decoding device accordingto the present embodiment.

The moving picture decoding device according to the present embodimentincludes, for example, a variable length decoding unit (302) whichperforms a procedure opposite to variable length coding to the videostream (301) generated by the moving picture encoding device shown inFIG. 1, an inverse quantization unit (303) which inversely quantizesresidual data, an inverse transform unit (304) which performs inversetransformation to the data inversely-quantized by the inversequantization unit (303), an inter prediction unit (305) which performsinter prediction, an intra prediction unit (306) which performs intraprediction, an addition unit (307) which generates a decoded image, anda reference frame memory (308) which stores the decoded image therein.

The operations of the respective component parts of the moving picturedecoding device according to the present embodiment will be explained.

The variable length decoding unit (302) performs variable lengthdecoding to the video stream (301) to acquire information necessary fora prediction process, such as a residual transform coefficientcomponent, a block size, motion vectors MV, etc. The residual transformcoefficient component and the like are transmitted to the inversequantization unit (303), and the block size and the motion vectors MVand so on are transmitted to the inter prediction unit (305) or theintra prediction unit (306). The inverse quantization unit (303) and theinverse transform unit (304) perform inverse quantization and inversetransformation to the residual information respectively to decode theresidual data. The inter prediction unit (305) and the intra predictionunit (306) perform a prediction process based on the image informationinputted from the variable length decoding unit (302) and the referenceframe stored in the reference frame memory (308). The addition unit(307) generates a decoded image based on the residual data inputted fromthe variable length decoding unit (302) and the prediction image datainputted from the inter prediction unit (305) or the intra predictionunit (306). The reference frame memory (308) stores the decoded imageinputted from the addition unit (307).

The inter prediction unit (305) will be described using FIG. 4. Theinter prediction unit (305) includes a motion vector memory (401) whichstores a motion vector MV of each decoded area, a prediction vectorcalculation unit (402) which calculates a prediction vector PMV usingthe motion vector MV of each decoded area, an addition unit (403) whichcalculates the sum of the difference vector DMV and the predictionvector PMV to calculate a motion vector MV, and a prediction imagecreation unit (404) which generates a prediction image.

The operations of the respective component parts of the inter predictionunit (305) will be explained.

The prediction vector calculation unit (402) calculates a predictionvector PMV of a target block, based on the motion vector MV of eachdecoded area stored in the motion vector memory (401). The addition unit(403) calculates the sum of the difference vector DMV decoded by thevariable length decoding unit (302) and the prediction vector PMV todecode the corresponding motion vector MV. The motion vector memory(401) stores the decoded motion vector MV therein. The prediction imagecreation unit (404) generates a prediction image (405) from the motionvector MV and the reference frame.

FIG. 12 is a diagram showing a procedure for encoding processing in thepresent embodiment.

The following processes are performed to all blocks that exist within aframe, which are targeted for encoding (1201). That is, prediction isperformed to all encoding modes (combination of a prediction method anda block size) for every target block (1202). Here, intra prediction(1204) or inter prediction (1220) is performed according to a predictionmode to carry out the calculation of a residual. Further, when the interprediction (1220) is performed, each motion vector MV is encoded inaddition to the residual (1205). A combining process is performed to themotion vectors MV of the encoded area (1206). It is determined whether acombined area exists around the target block (1207). If the combinedarea exists around the target block, a prediction vector PMV iscalculated based on the motion vectors MV included in the combined area(1208). A difference between the prediction vector PMV and itscorresponding motion vector MV is calculated to thereby obtain adifference vector DMV (1209). On the other hand, if the combined areadoes not exist around the target block, a prediction vector PMV iscalculated by the conventional method such as H. 264 (1210) to therebyacquire a difference vector DMV (1211). Subsequently, a transformprocess (1212), a quantization process (1213) and a variable lengthcoding process (1214) are performed to residual data to therebycalculate image-quality distortion and the amount of coded bits in eachencoding mode. If the aforementioned processing is completed withrespect to all the encoding modes, the encoding mode best in codingefficiency is selected based on the above result of processing (1215).Incidentally, for example, the RD-Optimization method for determiningthe optimal encoding mode from the relationship between theimage-quality distortion and the amount of coded bits is used to selectthe encoding mode best in coding efficiency out of the plural encodingmodes, thereby making it possible to perform encoding efficiently. Forthe details of the RD-Optimization method, refers to a ReferenceLiterature 1.

(Reference Literature 1)

-   G. Sullivan and T. Wiegand: “Rate-Distortion Optimization for Video    Compression”, IEEE Signal Processing Magazine, vol. 25, no. 6, pp.    74-90, 1998.

Subsequently, an inverse quantization process (1216) and an inversetransform process (1217) are performed to the quantized transformcoefficient in the selected encoding mode to decode the residual data,thereby generating a decoded image (1218), which in turn is stored inthe reference frame memory. If the aforementioned processing iscompleted with respect to all blocks, the encoding of an image of 1frame is ended (1219).

FIG. 13 is a diagram showing a procedure for the decoding of 1 frame inthe present embodiment.

The following processes are first performed to all blocks in one frame(1301). That is, a variable length decoding process is performed to aninput stream (1302). Then, an inverse quantization process (1303) and aninverse transform process (1304) are performed to decode residual data.Subsequently, an intra prediction process (1306) and an inter predictionprocess (1315) are performed according to a prediction mode.Incidentally, the decoding of each motion vector MV is performed uponexecution of inter prediction. Here, a combining process is performed toeach motion vector MV of the decoded area (1307). It is determinedwhether a combined area exists around a target block (1308). If thecombined area exists around the target block, a prediction vector PMV iscalculated based on the motion vectors MV included in the combined area(1309). The prediction vector PMV and its corresponding differencevector DMV are added to obtain a motion vector MV of the target block(1310). On the other hand, if the combined area does not exist aroundthe target block, a prediction vector PMV is calculated by theconventional method such as H. 264 (1311) and a motion vector MV iscalculated (1312). Then, the generation of a prediction image and adecoded image are performed using the calculated motion vector MV(1313). If the aforementioned processing is completed with respect toall the blocks in the frame, the decoding of an image of one frame iscompleted (1314).

Incidentally, in the present embodiment, the calculation of theprediction vector PMV is performed in block units. Even at other thanthe above, however, the prediction vector may be calculated in units ofobjects discrete from the background of an image, for example. AlthoughDCT has been taken as one example of the transformation, any of DST(Discrete Sine Transformation), WT (Wavelet Transformation), DFT(Discrete Fourier Transformation), KLT (Karhunen-Loeve Transformation),etc. may be adopted if it takes transform used in the elimination of aninter-pixel correlation. In particular, coding may be performed to aresidual itself without performing transformation. Further, the variablelength coding may not be performed. In the present embodiment, theprediction vector PMV is calculated based on the motion vectors MVincluded in the combined area, but each motion vector itself may becalculated using it.

In the present embodiment, when the combined area exists around thetarget block, the target block is also determined to include a part ofthe same object as that included in the combined area, and thecorresponding prediction vector is calculated based on motion vectors ofblocks included in the combined area. It is thus possible to improveprediction accuracy for each motion vector. The method of calculatingthe prediction vector is improved to make it possible to reduce theamount of coded bits for each motion vector and improve the efficiencyof compression.

Embodiment 2

In the embodiment 1, the prediction accuracy is improved where thetarget block is included in its corresponding peripheral object area. Onthe other hand, in the embodiment 2, a prediction vector PMV of a targetblock is calculated using encoded or decoded previous frames. It is thuspossible to improve prediction accuracy where a target block is locatedin the boundary of object areas, for example.

FIG. 9 is a diagram showing one example using encoded previous frames.

Here, the combining of motion vectors MV is performed to a frame(display time t=m) (902) preceding a target frame (display time t=n)(901) by a procedure similar to that in the embodiment 1. The frame(902) refers to a frame (display time t=k) (903) further preceding it,and a destination to move a combined area (904) in the frame (902) canbe represented by a typical value F(C) (905) of a motion vector MVincluded in the combined area. Here, the typical value F(C) may becalculated by the average value of the motion vectors MV included in thecombined area, a weighted average value, a median value and the like.Incidentally, assume k<m<n. Considering the values of k, m, and n atthis time, a correction vector F′ (C) (906) for estimating where thecombined area (904) is moved within the target frame (901) can becalculated using an equation (908), for example. Thus, a motionestimation area (907) on the target frame (903) corresponding to thecombined area (904) in the previous frame (902) can be estimated.

When the target block is included in the area (907), a prediction vectorPMV is calculated based on the motion vectors MV included in thecombined area (904). For example, the prediction vector PMV may becalculated as an average value AVE (C) of the motion vectors MV includedin the combined area (904), or one vector included in the combined area(904) may be selected. Alternatively, the prediction vector PMV may becalculated using the conventional method such as H. 264. Further, theprediction vector PMV can be computed by, for example, equations (909),(910), (911) and so on.

Since the configurations of the moving picture encoding device and themoving picture decoding device according to the present embodiment aresimilar to the configurations of the moving picture encoding device andthe moving picture decoding device shown in FIGS. 1 through 4 in theembodiment 1, the description of these is omitted.

FIG. 14 is a diagram showing a procedure of encoding in the presentembodiment. Only the inter prediction process (1220) shown in FIG. 12 isdescribed herein. Since other operations are taken to carry out aprocedure similar to FIG. 12, their description is omitted.

A motion vector calculation and a difference image calculation are firstconducted (1401). Subsequently, a combining process is performed tomotion vectors MV in a previous frame (1402), whereby a typical vector(effective if calculated as an average value vector, for example, andhereinafter be described as “average vector”) of a combined area iscalculated (1403). Then, a correction vector indicative of where acombined area in a previous frame is moved within a coding target frameis calculated based on time information on the average vector and theprevious frame (1404) to thereby specify a motion estimation area. It isdetermined whether a target block is included in the motion estimationarea (1405). If the target block is included in the motion estimationarea, a prediction vector PMV is computed based on the motion vectors MVincluded in the combined area (1406). A method for calculating theprediction vector PMV is as shown in FIG. 9. Next, a difference betweenthe prediction vector PMV and its corresponding motion vector MV iscalculated to obtain a difference vector DMV (1407). On the other hand,if the target block is not included in the motion estimation area, aprediction vector PMV is calculated by the conventional method such asH. 264 (1408) to obtain a difference vector DMV (1409).

FIG. 15 is a diagram showing a procedure of decoding in the presentembodiment. Only the inter prediction process (1315) shown in FIG. 13 isdescribed herein. Since other operations are taken to conduct aprocedure similar to FIG. 13, their description is omitted.

A combining process is performed to motion vectors MV in a previousframe (1501), and an average vector of a combined area is calculated(1502). Subsequently, a correction vector indicative of where a combinedarea in a previous frame is moved within a coding target frame iscalculated based on the times of the average vector and the previousframe (1503) to specify a motion estimation area. It is determinedwhether a target block is included in the motion estimation area (1504).If the target block is included in the motion estimation area, aprediction vector PMV is calculated based on the motion vectors MVincluded in the combined area (1505). The sum of the prediction vectorPMV and a difference vector DMV is calculated to obtain a motion vectorMV (1506). On the other hand, if the target block is not included in themotion estimation area, a prediction vector PMV is calculated by theconventional method such as H. 264 (1507). A method for calculating theprediction vector PMV is as shown in FIG. 9. Next, the sum of theprediction vector PMV and the difference vector DMV is computed tocalculate a motion vector MV (1508). Finally, a prediction image and adecoded image are generated based on the calculated motion vectors MV(1509), whereby image decoding is carried out.

Although the calculation of the prediction vector PMV is carried out inblock units in the present embodiment, the prediction vector may becalculated in units of objects discrete from the background of an image,for example, even at other than the above. Further, the presentembodiment and the embodiment 1 may be combined together.

In the present embodiment, the prediction vector PMV of the target blockis calculated using the encoded or decode previous frames. It is thuspossible to improve prediction accuracy where a target block is locatedin the boundary of object areas, for example.

Embodiment 3

In the embodiment 2, the destination (907) to move the combined area(904) was estimated using the correction vector (906) calculated by theaverage vector (905) of the combined vector (904). On the other hand, inthe embodiment 3, the motion of a combined area is modeled inmathematical expressions as shown in FIG. 10. If this operation isperformed on the encoding and decoding sides, it is then unnecessary toencode their motion parameters, thereby making it possible to preventthe amount of coded bits from increasing.

FIG. 10 is a diagram showing one example in which the motion of acombined area is modeled in mathematical expressions using encodedprevious frames.

From loci of combined areas at previous frames (1002), . . . , (1003),and (1004) of a target frame (1001), the motion thereof is modeled by atime function RMV(t). A motion estimation area (1006) in the targetframe (1001) is estimated by calculating RMV(n). As the function RMV(t)for modeling the motion, a model like a linear function such as will beindicated at (1008), for example, may be utilized. For example,functions such as an ellipse, a quadratic curve (parabola), a Beziercurve, a clothed curve, a cycloid, reflection, pendular movement, etc.may be utilized. At this time, motion parameters like A, B, C and D of(1008), for example, are required to perform the modeling of motion, butthey may be freely set on the encoding side so as to be capable of beingincluded in a stream. Alternatively, they may automatically becalculated from the loci of the combined areas. That is, if motionvectors MV are combined with respect to the respective previous frames(1002), . . . , (1003), and (1004), then a coordinate string (X1, Y1), .. . , (Xm−1, Ym−1), and (Xm, Ym) of combined areas can be obtained. Forthis reason, if these values are substituted into the equations such as(1008), and simultaneous equations are solved, the values of parameterscan be determined. If this operation is performed on the encoding anddecoding sides, there is no need to encode these motion parameters, thusmaking it possible to prevent the amount of coded bits from increasing.

Incidentally, the prediction vectors PMV in the motion estimation area(1006) calculated by the above function RMV(t) can be calculated by, forexample, equations (1009), (1010), (1011) and so on.

Since the configurations of the moving picture encoding device and themoving picture decoding device according to the present embodiment aresimilar to the configurations of the moving picture encoding device andthe moving picture decoding device shown in FIGS. 1 through 4 in theembodiment 1, the description of these is omitted.

FIG. 16 is a diagram showing a procedure of encoding in the presentembodiment. Only the inter prediction process (1220) shown in FIG. 12 isdescribed herein. Since other operations are taken to carry out aprocedure similar to FIG. 12, their description is omitted.

The calculation of each motion vector MV and a difference image is firstcarried out (1601). Next, the following processes are performed to allprevious frames to be targeted (1602). That is, a combining process isperformed to motion vectors MV of previous frames (1603). Typicalcoordinates of each combined area are calculated from the center ofgravity of the combined area, for example (1604). If a coordinate stringof the combined areas is obtained according to the above processing, ablock close in distance to a target block or a previous blockcorresponding to the target block, of coordinates of combined areasdetected in different frames, for example, is determined to be a blockincluding the same object as the target block, whereby a follow-upprocess of the combined area is performed (1605). That is, the follow-upprocess is of a process for determining the combined area determined toinclude the same object as the target block for every previous frame andgenerating the coordinate string comprised of typical coordinates of therespective combined areas. Subsequently, motion modeling parameters arecalculated from the coordinate string (1606). Using the calculatedparameters, a check is made where the combined area of each previousframe is moved within a target frame, thereby specifying a motionestimation area in the target frame (1607). It is determined whether thetarget block is included in the motion estimation area (1608). If thetarget block is included in the motion estimation area, a predictionvector PMV is calculated based on the motion vectors MV included in thecombined area (1609). A difference between the prediction vector PMV andits corresponding motion vector MV is computed to calculate a differencevector DMV (1610). On the other hand, if the target block is notincluded in the motion estimation area, a prediction vector PMV iscalculated by the conventional method such as H. 264 (1611), and adifference vector DMV is calculated (1612).

FIG. 17 is a diagram showing a procedure of decoding in the presentembodiment. Only the inter prediction process (1315) shown in FIG. 13 isdescribed herein. Since other operations are taken to carry out aprocedure similar to FIG. 13, their description is omitted.

The following processes are first performed to all previous frames to betargeted (1701). That is, a combining process is performed to motionvectors MV of each previous frame (1702). Typical coordinates of eachcombined area are calculated from the center of gravity of the combinedarea, for example (1703). If a coordinate string of the combined areasis obtained according to the above processing, a block close in distanceto a target block or a previous block corresponding to the target block,of coordinates of combined areas detected in different frames, forexample, is determined to be a block including the same object, wherebya follow-up process of the combined area is performed (1704).Subsequently, motion modeling parameters are calculated from thecoordinate string (1705). Using the calculated parameters, a check ismade where the combined area of each previous frame is moved within atarget frame, thereby specifying a motion estimation area (1706). It isdetermined whether the target block is included in the motion estimationarea (1707). If the target block is included in the motion estimationarea, a prediction vector PMV is calculated based on the motion vectorsMV included in the combined area (1708). The sum of the predictionvector PMV and a difference vector DMV is computed to calculate a motionvector MV (1709). On the other hand, if the target block is not includedin the motion estimation area, a prediction vector PMV is calculated bythe conventional method such as H. 264 (1710), and a motion vector MV iscalculated (1711). Finally, the generation of a prediction image and adecoded image is carried out based on the calculated motion vectors MV(1712).

Although the calculation of the prediction vector PMV is performed inblock units in the present embodiment, the prediction vector may becalculated in units of objects discrete from the background of an image,for example, even at other than the above. Further, the presentembodiment, the embodiment 1 and the embodiment 2 may be combinedtogether.

In the present embodiment, the motion of the combined area is modeled inmathematical equations as shown in FIG. 10. If this operation isperformed on the encoding and decoding sides, there is no need to encodemotion parameters of these, thus making it possible to prevent theamount of coded bits from increasing.

Embodiment 4

In the embodiments 1 through 3, the prediction vector PMV is calculatedusing the entire screen of each previous frame or the motion vectors MVof the entire encoded area in the current frame. On the other hand, theembodiment 4 shows a method capable of obtaining advantageous effectssimilar to those of the embodiments 1 through 3 with a small amount ofcoded bits within the same frame as a target block and using only itsperipheral blocks.

Since the configurations of the moving picture encoding device and themoving picture decoding device according to the present embodiment aresimilar to the configurations of the moving picture encoding device andthe moving picture decoding device shown in FIGS. 1 through 4 in theembodiment 1, the description of these is omitted.

FIG. 11 conceptually shows one example of a prediction vector PMVcalculating method by the present invention. Encoded blocks adjacent tothe left, upper and upper right sides of a target block (1101) areassumed to be a block A (1102), a block B (1103) and a block C (1104)respectively. Motion vectors in the respective blocks are assumed to beMVA, MVB and MVC respectively. At this time, a function Cluster forcalculating an inverse number of the degree capable of being combinedwith each peripheral vector is set to each vector MVX (MVXEMVA, MVB,MVC). The Cluster (MVX) is calculated using it, and a vector at whichthe value of the Cluster (MVX) is brought to a minimum, is selected(1105).

Here, the function Cluster is effective if a function for calculatingthe sum of absolute differences between the respective vectors MVX andtheir peripheral vectors is set to each vector MVX (MVX∈MVA, MVB, MVC).That is, assuming that encoded blocks adjacent to the left, upper andupper right sides of a block X (X∈A, B, C) (1107) are assumed to be ablock X1 (1108), a block X2 (1109) and a block X3 (1110) respectively,and motion vectors in the respective blocks are assumed to be MVX1, MVX2and MVX3 respectively, the function Cluster is expressed as designatedat (1111). The inner product of the motion vector MVX of the block X(X∈A, B, C) (1107) and each of motion vectors MVXn of the blocks X1, X2and X3 may be calculated based on a value divided by the product of theabsolute value of the motion vector of the block X (X∈A, B, C) (1107)and the absolute value of each of the motion vectors of the blocks X1,X2 and X3 (1112). If, however, there is provided one for calculatingsimilarity between the motion vectors, it may not comply with theseequations in particular.

Further, when the evaluation value Cluster (BESTMV) based on theselected vector is smaller than a constant Threshold2, it is judged thatan object exists around a target block, and a prediction vector PMV iscalculated based on the selected vector. This calculation method is notrestricted in particular, but it is effective if, for example, theselected vector is set to the prediction vector PMV as it is (1106). Onthe other hand, if the evaluation value Cluster (BESTMV) based on theselected vector is greater than Threshold2, the object is determined notto exist around the target block, and a prediction vector PMV iscalculated by a procedure similar to the conventional method such as H.264 (1106). The constant Threshold2 may be set to a uniform value inadvance. Alternatively, the value may be freely set on the encoding sideand included in a video stream. Further, the constant Threshold2 may bedetermined dynamically based on coding information about dispersedvalues of motion vectors MV and an average value thereof, the magnitudeof a combined area, etc. The constant Threshold2 may not comply withthis equation in particular.

The present embodiment is capable of obtaining the same advantageouseffect as where the motion vectors are combined with a small amount ofcoded bits, by using the above Cluster function. In the aboveembodiment, the block to which this function is applied is limited tothe three types of blocks on the left, right and upper right sides ofthe target block, but the number of blocks to which this function isapplied is not restricted in particular. Applying to four types ofblocks added with a block on the upper left side, for example, providesa further improvement in prediction accuracy.

FIG. 18 shows a procedure of encoding in the present embodiment. Onlythe inter prediction process (1220) shown in FIG. 12 is describedherein. Since other operations are taken to carry out a proceduresimilar to FIG. 12, their description is omitted.

The calculation of motion vectors and a difference image is firstcarried out (1801). Subsequently, a vector at which an evaluation valueCluster (MVx) indicative of similarity among motion vectors existingaround a target block and further with peripheral motion vectors becomesminimum, is selected (1802). The evaluation value Cluster (BESTMV) and athreshold are compared with each other (1803). If the evaluation valueCluster (BESTMV) is smaller than the threshold, the selected minimumvector is determined as a prediction vector PMV (1804), and a differencebetween the minimum vector and its corresponding motion vector MV iscomputed to calculate a difference vector DMV (1805). On the other hand,if the evaluation value Cluster (BESTMV) is greater than the threshold,a prediction vector PMV is calculated by the conventional method such asH. 264 (1806), and a difference vector DMV is calculated (1807).

FIG. 19 shows a procedure of decoding in the present embodiment. Onlythe inter prediction process (1315) shown in FIG. 13 is describedherein. Since other operations are taken to carry out a proceduresimilar to FIG. 13, their description is omitted.

First, a vector is selected at which an evaluation value Cluster (MVx)indicative of similarity among motion vectors existing around a targetblock and further with peripheral motion vectors becomes minimum (1901).The evaluation value Cluster (BESTMV) and a threshold are compared witheach other (1902). If the evaluation value Cluster (BESTMV) is smallerthan the threshold, the selected minimum vector is determined as aprediction vector PMV (1903), and the sum of the prediction vector PMVand a difference vector DMV is calculated, thereby obtaining a motionvector MV (1904). On the other hand, if the evaluation value Cluster(BESTMV) is greater than the threshold, a prediction vector PMV iscalculated by the conventional method such as H. 264 (1905), and therebya motion vector MV is calculated (1906). Finally, a prediction image anda decoded image are generated using the calculated motion vectors(1907), whereby image decoding is carried out.

The present embodiment utilizes only the blocks lying within the sameframe as the target block and at the periphery thereof. It is thuspossible to obtain advantageous effects similar to those of theembodiments 1 through 3 with a small amount of coded bits.

INDUSTRIAL APPLICABILITY

The present invention is effective as a moving picture decodingtechnique for decoding a moving picture and a moving picture encodingtechnique for encoding a moving picture.

REFERENCE SIGNS LIST

101 through 115 . . . explanatory diagram of moving picture encodingdevice according to the present invention, 201 through 207 . . .explanatory diagram of moving picture encoding device according to thepresent invention, 301 through 308 . . . explanatory diagram of movingpicture decoding device according to the present invention, 401 through406 . . . explanatory diagram of moving picture decoding deviceaccording to the present invention, 501 through 505 . . . explanatorydiagram of inter prediction encoding process by H. 264/AVC, 601 through606 . . . explanatory diagram related to motion vector predictiontechnique by H. 264/AVC, 701 through 706 . . . explanatory diagramrelated to motion vector prediction technique by the present invention,801 through 807 . . . explanatory diagram related to motion vectorprediction technique by the present invention, 901 through 911 . . .explanatory diagram related to motion vector prediction technique by thepresent invention, 1001 through 1011 . . . explanatory diagram relatedto motion vector prediction technique by the present invention, 1101through 1112 . . . explanatory diagram related to motion vectorprediction technique by the present invention, 1201 through 1219 . . .blocks of flowchart, 1301 through 1314 . . . blocks of flowchart, 1401through 1409 . . . blocks of flowchart, 1501 through 1509 . . . blocksof flowchart, 1601 through 1612 . . . blocks of flowchart, 1701 through1712 . . . blocks of flowchart, 1801 through 1807 . . . blocks offlowchart, and 1901 through 1907 . . . blocks of flowchart.

The invention claimed is:
 1. A moving picture decoding methodcomprising: a first step to receive encoded information of a targetblock to be decoded by an inter prediction process, the encodedinformation being encoded with a variable length coding but without afrequency transform process; a second step to decode predictiondifference data of the target block from the encoded information by avariable length decoding process without the frequency transformprocess; a third step to determine by which prediction mode the targetblock is encoded; a fourth step to generate a prediction image by theprediction mode determined by the third step, wherein if the targetblock is encoded by an intra prediction mode, a prediction image isgenerated by the intra prediction mode, wherein if the target block isencoded by a motion vector combined area mode, the prediction image isgenerated by using a motion vector determined so as to be in common witha motion vector of an adjacent block adjacent to the target blockwithout adding a difference vector, based on a status of motion vectorsof a plurality of adjacent blocks adjacent to the target block; andwherein a status of a motion vector of a previous frame is considered inthe determination of the motion vector to be used for the generation ofthe prediction image in the fourth step, and a fifth step to generate adecoded image of the target block based on the prediction imagegenerated in the fourth step and the prediction difference data decodedin the second step.
 2. A moving picture decoding method comprising: afirst step to receive encoded information of a target block to bedecoded by an inter prediction process, the encoded information beingencoded with a variable length coding but without a frequency transformprocess; a second step to decode prediction difference data of thetarget block from the encoded information by a variable length decodingprocess without the frequency transform process; a third step todetermine by which prediction mode the target block is encoded; a fourthstep to generate a prediction image by the prediction mode determined bythe third step, wherein if the target block is encoded by an intraprediction mode, a prediction image is generated by the intra predictionmode, wherein if the target block is encoded by a motion vector combinedarea mode, the prediction image is generated by using a motion vectordetermined so as to be in common with a motion vector of an adjacentblock adjacent to the target block without adding a difference vector,based on a status of motion vectors of a plurality of adjacent blocksadjacent to the target block; and wherein a status of a motion vector ofa previously decoded frame is considered in the determination of themotion vector to be used for the generation of the prediction image inthe fourth step, and a fifth step to generate a decoded image of thetarget block based on the prediction image generated in the fourth stepand the prediction difference data decoded in the second step.